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Abstract 

We introduce and study three new statistics on the hyperoctahedral group 
B n , and show that they give two generalizations of Carlitz's identity for the 
descent number and major index over S n . This answers a question posed by 
Foata. 



1 Introduction 



A well known classical result due to MacMahon (see jTB|) asserts that the inversion 
number and major index of a permutation are equidistributed over the symmetric 
group S n . The joint distribution of major index and descent number was studied 



by Carlitz [|J, Gessel JTTJ, and others. Despite the fact that an increasing number 
of enumerative results of this nature have been generalized to the hyperoctahedral 
group B n (see, e.g., ||, @, jl6fl , |17|| , and that several "major index" statistics 



have been introduced and studied for B n (see, e.g., ||, [[/J, ||14|, (T^|, p0|), 
no generalization of MacMahon's result has been found until the recent paper []]. 
There a new statistic, the flag major index (denoted here fmaj) was introduced; and 
it was shown to be equidistributed with length, which is the natural analogue of 
inversion number from a Coxeter group theoretic point of view. No corresponding 
"descent statistic" has been found that allows the generalization to B n of the well 
known Carlitz identity for descent number and major index on S n (see and also 
Theorem B72] below), a problem first posed by Foata H. 



Problem 1.1 (Foata) : Extend the ("Euler-Mahonian") bivariate distribution of 
descent number and major index to the hyperoctahedral group B n . 

The purpose of this paper is to introduce and study three new statistics on 
B n : the negative descent (denoted ndes); the negative major (nmaj); and the flag 
descent (fdes). When restricted to S n they reduce to descent number, major index, 
and twice descent number, respectively, and they solve the above problem. More 
precisely, we show that nmaj is equidistributed with length on B n , and that ndes 
is the "right" corresponding descent statistic needed to extend Carlitz's result to 
B n , thus answering Foata's question. Finally, we prove the surprising result that 
the pair of statistics (fdes, fmaj) is equidistributed with (ndes, nmaj) over B n , thus 
obtaining a second generalization of Carlitz's identity to B n . 
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The organization of the paper is as follows. In the next section we collect sev- 
eral definitions, notation, and results that will be used in the sequel. In section 3 
we introduce a new "descent set" for elements of B n and, correspondingly, a new 
"descent number" ndes and a new "major index" nmaj. It is shown that nmaj 
is equidistributed with length over B n (Proposition |3.1| ), and, moreover, that the 
pair {ndes, nmaj) gives a generalization of Carlitz's identity (Theorem |3.2j ) and thus 
solves Foata's problem. In section 4 we introduce a "flag analogue" of the descent 
number for B n , fdes, and show that it gives a second solution to Foata's prob- 
lem (Corollary f4.2|) . We then deduce that the pairs of statistics (ndes, nmaj) and 
(fdes,fmaj) are equidistributed over B n (Corollary |4.5| ). 



2 Notation, Definitions, and Preliminaries 

In this section we collect some definitions, notation and results that will be used in 
the rest of this paper. We let P d = {1,2,3, . . .}, N d = P U {0}, Z be the ring of 
integers, and Q be the field of rational numbers; for a GN we let [a] = f {1, 2, . . . , a] 
(where [0] = f 0). Given n, m G Z, n < m, we let [n,m] = f {n, n + 1, . . . , m}. For 
ScNwe write S = {ai, . . . , a r }< to mean that S = {a\, . . . , a r } and a% < . . . < a r . 
The cardinality of a set A will be denoted by \A\. More generally, given a multiset 
M = {l ai , 2 fl2 , . . . , r ar } we denote by |M| its cardinality, so \M\ = £[ =1 Given a 
statement P we will sometimes find it convenient to let 

x (p) A M [ lj if p is true ' 

I 0, if P is false. 

Given a variable q and a commutative ring R we denote by R[q] (respectively, 

the ring of polynomials (respectively, formal power series) in q with coefficients in 

R. For i G N we let, as customary, [i] q = f 1 + q + q 2 + . . . + q' 1 ^ 1 (so [0] q = 0). 

Given a sequence a = (oi, . . . ,a n ) G Z n we say that a pair G [n] x [n] is 
an inversion of cr if z < j and a* > %. We say that i G [n — 1] is a descent of a" 
if a, > a i+ i. We denote by inv(a) (respectively, des(a)) the number of inversions 
(respectively, descents) of a. We also let 

maj(a) = i 

{i: a,i>a i+1 } 

and call it the major index of a. 
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Given a set T we let S(T) be the set of all bijections n : T — > T, and >S n = f 5([n]). 
If cr G then we write a = 01 ... a n to mean that <r(«) = <7j, for z = 1, . . . , n. If 
a G then we will also write a in disjoint cycle form (see, e.g., p. 17), and 



will usually omit to write the 1-cycles of a. For example, if a = 365492187 then we 
also write a = (9, 7, 1, 3, 5) (2, 6). Given a, r G 5 n we let err = f cr o r (composition of 
functions) so that, for example, (1,2)(2,3) = (1,2,3). 

We denote by B n the group of all bijections n of the set [—n, n] \ {0} onto itself 
such that 

7r(— a) = — 7r(o) 

for all a G [— n,n] \ {0}, with composition as the group operation. This group is 
usually known as the group of "signed permutations" on [n], or as the hyperoctahedral 
group of rank n. We identify S n as a subgroup of B n , and B n as a subgroup of S 2n , 
in the natural ways. 

If 7r G B n then we write n = [oi, . . . , a n ] to mean that 7r(i) = for i = 1, . . . , n, 
and we let 

inv(n) = f inv(ai, . . . , a„), 
c?esyi(7r) = f des(ai, . . . , a n ), 

maj A (n) = maj(ai, . . . ,a„), (1) 



and 



Neg(n) d = {i G [n] : di < 0}, 



ne^(7r) d = |iVeg(7r)|. 



It is well known (see, e.g., 0, Proposition 8.1.3]) that B n is a Coxeter group with 
respect to the generating set {so, si, s>i, ■ ■ ■ , s n _i}, where 

s = f [-1,2, ...n) 



and 



s { = [1, 2, . . . , i — 1, i + 1, i, i + 2, . . . n] 



for i — 1, . . . ,n — 1. This gives rise to two other natural statistics on £? n (similarly 
definable for any Coxeter group), namely 

£(•71") = f min{r G N : n = . . . Si r for some i±, . . . , i r G [0,n — 1]} 
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(known as the length of tt) and 

des B (n) = \{i G [0,n- 1] : /(vrs,) < /(tt)}|. 

There is a well known direct combinatorial way to compute these two statistics 
for 7r G B n (see, e.g., |2], Propositions 8.1.1 and 8.1.2] or |3], Proposition 3.1 and 
Corollary 3.2]), namely 

1{tx) = inv(n) — 7r(z) (2) 

i£Neg(ir) 

and 

c?es B (7r) = \{i G [0, n - 1] : > tx(i + 1)}|, (3) 

where 7r(0) = f 0. For example, if 7r = [—3, 1, —6, 2, —4, —5] G i?6 then inv(ir) = 9, 
(iesA(vr) = 3, maj A {n) = 11, neg(ii) = 4, /(7r) = 27, and dessi^) = 4. 
Let 

{7rGS n : des A (n) = 0}. (4) 
It is then well known, and easy to see, that 

B n = |+J {nu : tt G T}, (5) 

where 1+) denotes disjoint union. 

We will use this decomposition often in this paper. The reader familiar with 
Coxeter groups will immediately recognize that @ is one case of the multiplicative 
decomposition of a Coxeter group into a parabolic subgroup and its minimal coset 



representatives (see, e.g., [0 or 0). 

As customary, given a variable t we define an operator 5 t : Q[q, t] Q[q, t] by 

c, p( ,^ P(q,qt)-P(q,t) 

5 t (P{q,t)) = , 

qt — t 

for all P G Q[q,t}. Note that 5 t {q n ) = and 

8 t (t n ) = [n] q t n - 1 (6) 

for all n G N, and 

S t (A(q, t) B(q, t)) = 5 t (A(q, t))B(q, t) + A(q, qt)5 t (B(q, t)) (7) 
for all A,B<E Q[q,t]. Also, 

5 t (P)(l,t) = j t (P(l,t)) 



for all P G Q[q,t\. 
For n£P we let 

A n (t,q) = t deSA{a) q maj ^ a \ 



clef 

cr£S„ 

def 



and Ao(t, q) = 1. For example, ^(t, i]) = 1 + 2tq 2 + 2tq + t 2 q 3 . The following two 
results are due to Carlitz || and Gessel flH|| , and proofs of them can also be found 



in 0. 



Theorem 2.1 Let n G P. T/ien 

A„(t, cj) = (1 + ig[n - 1],) A n ^(t, q) + tq(l - t)5 t (A n ^(t, q)). 

Theorem 2.2 Let n G P. T/jen 



i=0 



inZ[q][[t]}. 

3 The "Negative" Statistics 

In this section we define and study a new "descent set" for the elements of B n . 
This gives rise, in a very natural way, to new "major index" and "descent number" 
statistics for B n . We then show that these two statistics give a generalization of 
Carlitz's identity to B n , and that one of them is equidistributed with length. 

3.1 The Negative Descent Multiset 

For 7v G B n we define 

Des A (ir) = {i G [n - 1] : ir{i) > n(i + 1)} 
and the negative descent multiset 

NDes(n) = Des A (n) l+J{-7r(i) : i G Neg(n)}, (8) 
where Neg(7r) is the set of positions of negative entries in n, defined in (|T|). 
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For example, if ix = [—3,1,-6,2,-4,-5] G B e then DesAijt) = {2,4,5} and 
NDes{n) = {2, 3, 4 2 , 5 2 , 6}. Note that if 7r G S n then NDes(ir) is a set and coincides 
with the usual descent set of 7r. Also note that NDes(n) can be defined rather 
naturally also in purely Coxeter group theoretic terms. In fact, for i G [n] let 
r/i G -B n be defined by 

r/i d = [1,2, . . . ,i - 1, + 1, . . . ,n], 

so 771 = s - Then rji,...,r] n are reflections (in the Coxeter group sense; see, e.g., 
fj~2|| ) of B n and it is clear from (Q) that 

NDes(n) = {i £ [n - 1] : /(tts;) < /(tt)}|+J{2 G [n] : /(tT 1 ^) < /(t^ 1 )}. 

These considerations explain why it is natural to think of NDes(ir) as a "descent 
set". With this in mind, the following definitions are also natural. For n G B n we 
let 

ndes(n) = f \NDes(rr)\ 

and 

nmaj{n) = f i. 

i6iVZ>es(7r) 

For example, if tt = [—3, 1, —6, 2, —4, —5] G -E?6 then ndesijx) = 7 and nmajiji) = 29. 
Note that from (§) it follows that 

nmajiit) = maj A (7r) — 7r(z) (V7r G _B„). (9) 

This formula is also one of the motivations behind our definition of nmaj(7f), because 
of the corresponding formula (|2]) . 

3.2 Equidistribution and Generating Function 

Our first result shows that nmaj and / are equidistributed in B n . 

Proposition 3.1 Let nGP. Then 
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Proof. Let T be denned by (|j). It is clear from our definitions that for all u G S n 
and a G T, 

maj A (au) = maj A {u), inv(au) = inv(u), 

and 

E HW- E *(<)■ 

i£Neg(au) i&Neg(a) 

Therefore, from (Q), (|5|), (Q), and the corresponding classical result for S n (see, e.g., 
[P~3|]) we conclude that 



= E g majA(<T " )_ ^ lSJVe 9( CT ") (< ™ )(i) 

= y~] q~^ieNeg^) a ^ . V" A (u) 



= E E g^™)"^^^^^) 

cr£T u£Sn 

7TgB n 

as desired. □ 

We now prove the main result of this section, which is also the first main result 
of this work, namely that the pair of statistics (ndes, nmaj) gives a generalization 
of Theorem to B n . 



Theorem 3.2 Let n G P. Then 

' j-ndes(n) qnmaj(w) 
S[r+1] » f r = J[§5-__ 



r -° (i-t)n(i-^? 2i ) 

Proof. Let T have the same meaning as in Then it is clear from our definitions 
that desman) = des^u) for all a G T and u E S n . Therefore we have from (|5|) that 

t ndes(-K) qnmaj{-K) _ ^ ^ ^esA(cr«)+ne9(^) ? ma JA(^)-E 1 :eiv< i9 ( CTU )( fT ")( i ) 
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j-neg{<r)q-J2iENeg{«) a ( i } . ^des A (u) ^maj A (u) 

o-eT ues„ 



SC[n] ueS n 
n 

Iftl+tg*) ■ t deSA{u) q 

i=l u£S n 



des A (u) maj A (u) 



and the result follows from Theorem 2.2. □ 



4 The Flag Statistics 



In this section we introduce yet another pair of statistics on B n , and show that it 
also gives a solution to Foata's problem. We then derive some consequences of this 
result. 

4.1 The Flag Major Index 

For i = 0, 1, . . . , n — 1 let U = f ■ ■ ■ so. Explicitly, 

U= [-i - 1,1,2, ... + 2, ... ,n] 

for i — 0, . . . , n — 1. It is not hard to show (see also 0) that for any it 6 B n there 
exist unique integers fc , . . . , A; n _ 1; with < fcj < 2i + 1 for i = 0, . . . , n — 1, such 
that 

+ fcn-l j.fc2J.fclJ.feo 

" — t n-l • • • ( '2 n ''O • 

The /Zag major index of 7r (see is then defined by 

n-l 

fmaj(ir) = 

i=0 

There is a simple way to compute the flag major index of a signed permutation n. 
In fact, we have the following result which was first proved in Jlp. 

Theorem 4.1 Let % e B n . Then 

fmaj(n) = 2maj A (n) + neg(ir). 

For example, if tt = [-3, 1, -6, 2, -4, -5] then fmaj(n) = 2 • 11 + 4 = 26. 

In (]]] it was shown that fmaj appears naturally in the Hilbert series of (diagonal 
action) invariant algebras; note that this property is not shared by nmaj. 



4.2 The Flag Descent Number 

For 7r G B n let 



fdes{n) = 2 des A {n) + £i(tt), (10) 
where des^ is as in Section 2 and 

£l (7r) d J: f ( if ^ < °' (11) 
I 0, otherwise. 

This definition is motivated by Theorem |4.1| , and because of this similarity we call 
fdes(7r) the flag descent number of n. For example, if 7r = [—3, 1, —6, 2, —4, —5] then 
fdesijx) = 2-3 + 1 = 7. Note that from (|]), ([[]) and (JTTj) it follows immediately that 

fdes{n) = desA^Tf) + dessi^) (12) 

and also 

fdesiji) = des(n(—n), . . . , 7r(— 1), vr(l), . . . , vr(n)) 

for all 7r G B„. 

Our aim is to show that the pair of statistics (fdes,fmaj) gives a solution to 
Foata's problem and thus has the same joint distribution, over B n , as the pair 
(ndes, nmaj) defined in the previous section. 

4.3 Main Theorems 

The main result of this section is 

Theorem 4.2 Let n G P. Then 

tfdes(ir)qfmaj(n) 

£[r + V = ^^-n 

r -° (i - on( i - *v<) 

1=1 

inZ[q][[t]\. 

This solves Foata's problem and implies, together with Theorem |3.2| , that the 
two pairs of statistics (fdes,fmaj) and (ndes, nmaj) are equidistributed over B n . 

Theorem [4.2| will be proved in several steps. 
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Let, for convenience, 

S n (t,q)= ]T ^Wg^'W (13) 

TTeBn 

for all n G P, and set So(t,q) = f 1. For example, Si(t,q) = 1 + tq and S^t, g) = 
1 + 2tg + t<? 2 + t 2 g 2 + 2t 2 g 3 + t 3 q 4 . 

Our first result gives a recursion satisfied by the polynomials S n (t, q). 

Theorem 4.3 Let n G P. Then 

S n (t, q ) = (l+tq + t 2 q 2 [2n - 2],)S n _i(t, g) + tg(l - t)(l + tq)S t (S n - 1 (t, q)). 
Proof. The result is clear for n < 2, so fix n > 3. For a G 5 n _i and z G [n] let 
<7i = f [cr(l), . . . , a(i - l),n, a(i), ...,a(n- 1)] 



and 



a_i d = [cr(l), . . . , <r(i - 1), -n, o-(i), . . . , a(n - 1)]. 



Then clearly 

n 

crS-Bn-l *=1 

Using (|I2] ) and our definitions it is not hard to check (though with some patience) 
that, for all a G -B n -i, 

fdes{a x ) = fdes(a) + 1 + > 0]; 

fdes(a^) = fdes(a) + X W(1) > 0]; 
fdes(a ±i ) = fdes(a) + 2 X [<j{i - 1) < <r(«)] 

for z = 2, . . . , n — 1; 

fdes(a n ) = fdes(a); 

and 

fdes(a^ n ) = fdes(a) + 2. 

Similarly, let 

di(a) d ^ \{j G Des A (a) : j > i}\ (i = 1, . . . , n - 1) 
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be the number of descents in a from position i on. Then 

maj A (ai) = maj A (a) + d^a) + (i - 1) xW(i ~ 1) < <*(«)] + 1 

and 

maj A (a_i) = maj A {a) + di(a) + (i - 1) - 1) < 
for z = l,...,n — 1; 

maj A (a n ) = maj A (a), 

and 

maj A (a^ n ) = maj A (a) + n-l. 
Therefore, using Theorem [4.1| , 



L/<ies((T i ) qfmaj(ai) _|_ ^fdes{o-i) ^/maj(o"-i) 



i=l 



_ ^/(ies(<j)+x[<T(l)>0] ^ai(a)+2di(<j)+l/ i? + 

+ t /deS ^ g/ ma J'( CT )+ 2 *( ,J )+ 1 (g -f ]_) + 

{ie[2,n-l]: <r(i-l)>o-(i)} 
+ ^ t fdes(a)+2 q fmaj(a)+2d 1 (a)+2i-l^ + ^ 

{ie[2,n-l]: o-(i-l)<a(j)} 



From the definition of <ij(<r) it is clear that 



^(,7) = ? 2(fc-i) 

{ie[2,n-l]: cr(i-l)>cr(j)} fc=l 

for all a 6 B n _i. On the other hand, let 

{ii, i 2 , • • • , ia}< = {i E[2,n- 1] : a(i - 1) < cr(i)}, 
so that a = n — 2 — des^o"). Then 

a(i a ) > a(i a + 1) > . . . > a(n - 1), 

and therefore 

i a + d ia (a) = i a + (n - 1 - i a ) = n - 1. 
Similarly, for each j 6 [o - 1] one has 

a(ij) > a(ij + 1) > . . . > a(i j+1 - 1) < 
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and therefore 

ij + di 3 {a) = ij + d ij+1 (a) + - ij - 1) = + d ij+1 {a) - 1. 
This shows that 

q 2(di(a)+i) _ q 2(n-k) ^ 
{ie[2,n-l]: <r(i-l)<cr(i)} k=l 

for all a G £? n -i. 

From (Tl5|), (Tl6|), and ( [TT]) we conclude that 

n 
i=l 

_ ^/des(t7-)+x[o-(l)>0] ^/maj(o-)+2des A (o-)+l^ _j_ ^ \ _j_ 

desA (°") 

£/des(CT)^wa?(<7)+l^ _j_ ^ g2(fc-l) _j_ 

fc=l 



_|_ ^/des(a)+2^/maj(<T)-l^ _|_ ^ q 2(n-k) _|_ 

fc=l 

_|_ j-fdes (a - ) qfmaj (<x) / _|_ ^q 2 ^ 1 ) = 
_ j.fdes{a)qfmaj(a) |^xk(l)>0] ^2des A (ff)+l ^ _|_ 
2^^4(0-) 2n-2 

+ E ^' + * 2 E ^ + (i + tV n ^)} 

j=l j=2n-2a-l 
__ £fdes(a)qfmaj(cr) |^xk(l)>0] ^2des A (a) + l /j _|_ ^ \_|_ 



2cies^(<j) 2n-l 

+ E ^' + * 2 E 

j=0 j=2ra-2a-l 

Using the value of a, fllDD and some case-by-case analysis (depending on the sign of 
cx(l)) we get 



it 

^es(o-i) qfmajicri) _|_ ^fdes(cr^i) ^fmaj^C-i) _ ^g^ 

t=l 

= t fdes{a) q fma]ia) { [fdes(tr) + 1], + * g > a "M+i + 
+ t 2 ([2n],-[/de S (a) + 2] (? )} = 

= ^W^'W {1 + g[/tfes(<r)], + tg + tg(g - l)[fdes(a)] q + 
+ t 2 q 2 ([2n-2] q -\fdes(a)] q )}. (19) 
Substituting ( [19|) into flllD we now obtain 

S n (t,g) = (l + tg + ty[2n-2],)S„_i(i,g) + 

+ (q + tq(q- 1) -t 2 g 2 ) E L/aesCa)],^^^^, 
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and the result follows from (|13|) and (^). □ 

Using the previous theorem we can now prove the following result. 

Theorem 4.4 Let n G P. Then 

n 

£fdes(Tr)qfmaj(iT) _ TJ(1 + tg*) • ^ £ des a{&) grriaj A (o) 
■K(zB n i=l <y£S n 

Proof. We show that both sides satisfy the same recursion; equality of initial 
conditions is easily checked (n < 2). Let, for convenience, 



S n (t,q)^l[(l+tq i )-A n (t,q) 



def 
i=l 

be the right-hand side of the formula in Theorem [O . 
It is easily verified that 



5 t m(l+tq*))=q[n} q Y[(l+tq*), 



\i=l / i=2 

so that by (0) 

n+l 

(1 + tq)5 t (S n (t, q)) = q[n] q S n {t, q) + J[ (1 + tg*) • 5 t (A n (t, g)). 

i=l 



By Theorem |2J] we therefore conclude that 

n 

Sn(t,q) = Y[(l+tq l )-A n (t iq ) = 

i=i 

n 

= n(! + t( f) ■ {(! + - !]?) A*-i(«, + - *)<S*(A»-i(*, g))} 
i=i 

= (l + tg ft )(l + *g[ra-l],)£ n _i(*,g) + 

+ - t){(l + tq)5 t (S n - 1 (t, q)) - q[n - l] q S n -i(t, q)} = 

= (1 + tq + £ 2 g 2 [2rc - 2],)5 n _ 1 (t, g) + tg(l - t)(l + tq)8 t (S n ^(t, q)), 



and comparison with Theorem [4.3| completes the proof. □ 

Theorem [O] has several interesting consequences. The first one is that the 
descent statistic fdes yields a solution to Foata's problem (Theorem |4.2p . 

Proof of Theorem [4.2| . This follows immediately from Theorems [4.4| and |2.2j , 

given the definition of A n (t, q). □ 



This in turn implies, together with Theorem |3.2| , that the two pairs of statistics 
(fdes,fmaj) and (ndes, nmaj) are equidistributed in B n . 
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Corollary 4.5 Let n G P. Then 



prides (n) q'nmaj(n) _ ^fdes(n) ^fmaj (n) 

ir£B n neB n 

It would be interesting to have a direct combinatorial (i.e., bijective) proof of 
this result. 

Finally, the special case t — 1 of Corollary [4.5| , together with Proposition |3.1| , 
implies the following result, which extends Theorem 2.2 of ||. 

Corollary 4.6 Let n G P. T/ien 
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